Scaling up the task-sharing of psychological therapies: A formative study of the PEERS smartphone application for supervision and quality assurance in rural India.

Measurement-based peer supervision is one strategy to assure the quality of psychological treatments delivered by non-mental health specialist providers. In this formative study, we aimed to 1) describe the development and 2) examine the acceptability and feasibility of PEERS (Promoting Effective mental healthcare through peER Supervision)-a novel smartphone app that aims to facilitate registering and scheduling patients, collecting patient outcomes, rating therapy quality and assessing supervision quality-among frontline treatment providers delivering behavioral activation treatment for depression. The PEERS prototype was developed and tested in 2021, and version 1 was launched in 2022. To date, 215 treatment providers (98% female; ages 30-35) in Madhya Pradesh and Goa, India, have been trained to use PEERS and 65.58% have completed the supplemental, virtual PEERS course. Focus group discussions with 98 providers were examined according to four themes-training and education, app effectiveness, user experience and adherence and data privacy and safety. This yielded commonly endorsed facilitators (e.g., collaborative learning through group supervision, the convenience of consolidated patient data), barriers (e.g., difficulties with new technologies) and suggested changes (e.g., esthetic improvements, suicide risk assessment prompt). The PEERS app has the potential to scale measurement-based peer supervision to facilitate quality assurance of psychological treatments across contexts.

Designing an App to Support Measurement-Based Peer Supervision of Frontline Health Workers Delivering Brief Psychosocial Interventions in Texas: Multimethod Study.

BACKGROUND: The unmet need for mental health care affects millions of Americans. A growing body of evidence in implementation science supports the effectiveness of task sharing in the delivery of brief psychosocial interventions. The digitization of training and processes supporting supervision can rapidly scale up task-shared interventions and enable frontline health workers (FLWs) to learn, master, and deliver interventions with quality and support. OBJECTIVE: We aimed to assess the perceived feasibility and acceptability of a novel mobile and web app designed and adapted to support the supervision, training, and quality assurance of FLWs delivering brief psychosocial interventions. METHODS: We followed human-centered design principles to adapt a prototype app for FLWs delivering brief psychosocial interventions for depression, drawing from an app previously designed for use in rural India. Using a multimethod approach, we conducted focus group sessions comprising usability testing and group interviews with FLWs recruited from a large health system in Texas to assess the feasibility and acceptability of the app. The positive System Usability Scale was used to determine the app's overall usability. We also calculated the participants' likelihood of recommending the app to others using ratings of 0 to 10 from least to most likely (net promoter score). Focus group transcripts were coded and analyzed thematically, and recommendations were summarized across 4 key domains. RESULTS: A total of 18 FLWs varying in role and experience with client care participated in the study. Participants found the app to be usable, with an average System Usability Scale score of 72.5 (SD 18.1), consistent with the industry benchmark of 68. Participants' likelihood of recommending the app ranged from 5 to 10, yielding a net promoter score of 0, indicating medium acceptability. Overall impressions of the app from participants were positive. Most participants (15/18, 83%) found the app easy to access and navigate. The app was considered important to support FLWs in delivering high-quality mental health care services. Participants felt that the app could provide more structure to FLW training and supervision processes through the systematic collection and facilitation of performance-related feedback. Key concerns included privacy-related and time constraints regarding implementing a separate peer supervision mechanism that may add to FLWs' workloads. CONCLUSIONS: We designed, built, and tested a usable, functional mobile and web app prototype that supports FLW-delivered psychosocial interventions in the United States through a structured supervision mechanism and systematic collection and review of performance measures. The app has the potential to scale the work of FLWs tasked with delivering these interventions to the hardest-to-reach communities they serve. The results of this project will inform future work to evaluate the app's use and efficacy in real-world settings to support task-shared mental health programs across the United States.

Chatbots That Deliver Contraceptive Support: Systematic Review.

BACKGROUND: A chatbot is a computer program that is designed to simulate conversation with humans. Chatbots may offer rapid, responsive, and private contraceptive information; counseling; and linkages to products and services, which could improve contraceptive knowledge, attitudes, and behaviors. OBJECTIVE: This review aimed to systematically collate and interpret evidence to determine whether and how chatbots improve contraceptive knowledge, attitudes, and behaviors. Contraceptive knowledge, attitudes, and behaviors include access to contraceptive information, understanding of contraceptive information, access to contraceptive services, contraceptive uptake, contraceptive continuation, and contraceptive communication or negotiation skills. A secondary aim of the review is to identify and summarize best practice recommendations for chatbot development to improve contraceptive outcomes, including the cost-effectiveness of chatbots where evidence is available. METHODS: We systematically searched peer-reviewed and gray literature (2010-2022) for papers that evaluated chatbots offering contraceptive information and services. Sources were included if they featured a chatbot and addressed an element of contraception, for example, uptake of hormonal contraceptives. Literature was assessed for methodological quality using appropriate quality assessment tools. Data were extracted from the included sources using a data extraction framework. A narrative synthesis approach was used to collate qualitative evidence as quantitative evidence was too sparse for a quantitative synthesis to be carried out. RESULTS: We identified 15 sources, including 8 original research papers and 7 gray literature papers. These sources included 16 unique chatbots. This review found the following evidence on the impact and efficacy of chatbots: a large, robust randomized controlled trial suggests that chatbots have no effect on intention to use contraception; a small, uncontrolled cohort study suggests increased uptake of contraception among adolescent girls; and a development report, using poor-quality methods, suggests no impact on improved access to services. There is also poor-quality evidence to suggest increased contraceptive knowledge from interacting with chatbot content. User engagement was mixed, with some chatbots reaching wide audiences and others reaching very small audiences. User feedback suggests that chatbots may be experienced as acceptable, convenient, anonymous, and private, but also as incompetent, inconvenient, and unsympathetic. The best practice guidance on the development of chatbots to improve contraceptive knowledge, attitudes, and behaviors is consistent with that in the literature on chatbots in other health care fields. CONCLUSIONS: We found limited and conflicting evidence on chatbots to improve contraceptive knowledge, attitudes, and behaviors. Further research that examines the impact of chatbot interventions in comparison with alternative technologies, acknowledges the varied and changing nature of chatbot interventions, and seeks to identify key features associated with improved contraceptive outcomes is needed. The limitations of this review include the limited evidence available on this topic, the lack of formal evaluation of chatbots in this field, and the lack of standardized definition of what a chatbot is.

Using a Responsive Feedback Approach to Develop and Pilot a Counseling Chatbot to Strengthen Child Nutrition in Rural India.

INTRODUCTION: Nearly half of all deaths in children in India aged younger than 5 years are attributable to undernutrition. Reinforcing the caregiver's positive behaviors through multiple channels can lead to improved child nutrition outcomes. We describe the development and piloting of a chatbot to improve nutrition outcomes for children aged 0-12 months. POSHAN DIDI CHATBOT DEVELOPMENT PROCESS: We engaged key stakeholders to understand existing interventions to improve nutrition outcomes and developed a theory of change that included an intervention that provides nutrition-related information directly to beneficiaries. A chatbot, Poshan Didi, was developed to provide individual counseling to mothers with children aged 0-12 months on age-appropriate, nutrition-related topics. The chatbot was piloted in Katni district from February 2019 to October 2019 in 2 phases to investigate the acceptability and feasibility of the chatbot (Phase 1, n=10 mothers) and to assess whether users would continue to be engaged and would engage with both the automated content as well as through free form questions (Phase 2, n=100 mothers). Quantitative and qualitative data were collected in focus group discussions with health care workers and mothers (n=4) and interviews with mothers (n=26). RESULTS: Mothers viewed the chatbot as being a private channel to discuss topics and referred to Poshan Didi's persona as being knowledgeable. Eighty percent of users replied at least once to a chatbot-generated message. Sixty-four percent engaged beyond the standard chatbot content by discussing at least 1 issue with a nurse who responded to escalated messages. CONCLUSION: This work demonstrates how chatbots could create multiple interaction touchpoints between caregivers and health care workers to increase caregivers' access to age-appropriate nutrition counseling and information. The study shows the value of using the responsive feedback approach in the chatbot design and implementation to improve the efficacy of the digital tool.

Chatbots to Improve Sexual and Reproductive Health: Realist Synthesis.

BACKGROUND: Digital technologies may improve sexual and reproductive health (SRH) across diverse settings. Chatbots are computer programs designed to simulate human conversation, and there is a growing interest in the potential for chatbots to provide responsive and accurate information, counseling, linkages to products and services, or a companion on an SRH journey. OBJECTIVE: This review aimed to identify assumptions about the value of chatbots for SRH and collate the evidence to support them. METHODS: We used a realist approach that starts with an initial program theory and generates causal explanations in the form of context, mechanism, and outcome configurations to test and develop that theory. We generated our program theory, drawing on the expertise of the research team, and then searched the literature to add depth and develop this theory with evidence. RESULTS: The evidence supports our program theory, which suggests that chatbots are a promising intervention for SRH information and service delivery. This is because chatbots offer anonymous and nonjudgmental interactions that encourage disclosure of personal information, provide complex information in a responsive and conversational tone that increases understanding, link to SRH conversations within web-based and offline social networks, provide immediate support or service provision 24/7 by automating some tasks, and provide the potential to develop long-term relationships with users who return over time. However, chatbots may be less valuable where people find any conversation about SRH (even with a chatbot) stigmatizing, for those who lack confidential access to digital devices, where conversations do not feel natural, and where chatbots are developed as stand-alone interventions without reference to service contexts. CONCLUSIONS: Chatbots in SRH could be developed further to automate simple tasks and support service delivery. They should prioritize achieving an authentic conversational tone, which could be developed to facilitate content sharing in social networks, should support long-term relationship building with their users, and should be integrated into wider service networks.

Implementing a digital system for contact tracing and case investigation during COVID-19 pandemic in San Francisco: a qualitative study.

During the COVID-19 pandemic, many health jurisdictions deployed digital informatics systems to support "manual" case investigation and contact tracing (CICT). This case study evaluates the implementation and use of a digital information system through the experiences of CICT workers in the City and County of San Francisco (CCSF). We conducted semi-structured, 90-min interviews with a sample of the CCSF CICT workforce (n = 37). Participants also completed standardized assessments of the digital system using the System Usability Scale (SUS). Qualitative analyses highlighted (1) the importance of digital tools to ensure rapid onboarding and effective data capture in a public health emergency; (2) the use of digital systems to support culturally sensitive care; and (3) the role of digitals tools in building supportive work environments. The mean SUS score was 70/100 (SD = 17), indicating relative ease of use. In summary, the analysis highlights the importance of digital tools to support manual CICT in the COVID-19 response.

Effect of Performance Feedback on Community Health Workers' Motivation and Performance in Madhya Pradesh, India: A Randomized Controlled Trial.

BACKGROUND: Small-scale community health worker (CHW) programs provide basic health services and strengthen health systems in resource-poor settings. This paper focuses on improving CHW performance by providing individual feedback to CHWs working with an mHealth program to address malnutrition in children younger than 5 years. OBJECTIVE: The paper aims to evaluate the immediate and retention effects of providing performance feedback and supportive supervision on CHW motivation and performance for CHWs working with an mHealth platform to reduce malnutrition in five districts of Madhya Pradesh, India. We expected a positive impact on CHW performance for the indicator they received feedback on. Performance on indicators the CHW did not receive feedback on was not expected to change. METHODS: In a randomized controlled trial, 60 CHWs were randomized into three treatment groups based on overall baseline performance ranks to achieve balanced treatment groups. Data for each treatment indicator were analyzed with the other two treatments acting as the control. In total, 10 CHWs were lost to follow-up. There were three performance indicators: case activity, form submissions, and duration of counseling. Each group received weekly calls to provide performance targets and discuss their performance on the specific indicator they were allocated to as well as any challenges or technical issues faced during the week for a 6-week period. Data were collected for a further 4 weeks to assess intertemporal sustained effects of the intervention. RESULTS: We found positive and significant impacts on duration of counseling, whereas case activity and number of form submissions did not show significant improvements as a result of the intervention. We found a moderate to large effect (Glass's delta=0.97, P=.004) of providing performance feedback on counseling times in the initial 6 weeks. These effects were sustained in the postintervention period (Glass's delta=1.69, P<.001). The counseling times decreased slightly from the intervention to postintervention period by 2.14 minutes (P=.01). Case activity improved for all CHWs after the intervention. We also performed the analysis by replacing the CHWs lost to follow-up with those in their treatment groups with the closest ranks in baseline performance and found similar results. CONCLUSIONS: Calls providing performance feedback are effective in improving CHW motivation and performance. Providing feedback had a positive effect on performance in the case of duration of counseling. The results suggest that difficulty in achieving the performance target can affect results of performance feedback. Regardless of the performance information disclosed, calls can improve performance due to elements of supportive supervision included in the calls encouraging CHW motivation.

Adoption and Usage of mHealth Technology on Quality and Experience of Care Provided by Frontline Workers: Observations From Rural India.

BACKGROUND: mHealth apps are deployed with the aim of improving access, quality, and experience of health care. It is possible that any mHealth intervention can yield differential impacts for different types of users. Mediating and determining factors, including personal and socioeconomic factors, affect technology adoption, the way health workers leverage and use the technology, and subsequently the quality and experience of care they provide. OBJECTIVE: To develop a framework to assess whether mHealth platforms affect the quality and experience of care provided by frontline workers, and whether these effects on quality and experience are different depending on the level of technology adoption and individual characteristics of the health worker. Literacy, education, age, and previous mobile experience are identified as individual factors that affect technology adoption and use, as well as factors that affect the quality and experience of care directly and via the technology. METHODS: Formative research was conducted with 15 community health workers (CHWs) using CommCare, an mHealth app for maternal and newborn care, in Bihar, India. CHWs were first classified on the level of CommCare adoption using data from CommCareHQ and were then shadowed on home visits to evaluate their levels of technology proficiency, and the quality and experience of care provided. Regression techniques were employed to test the relationships. Out of all the CHWs, 2 of them refused to participate in the home visits, however, we did have information on their levels of technology adoption and background characteristics, which were included in the analysis as relevant. RESULTS: Level of technology adoption was important for both quality and experience of care. The quality score for high users of CommCare was higher by 33.4% (P=.04), on average, compared to low users of CommCare. Those who scored higher on CommCare proficiency also provided significantly higher quality and experience of care, where an additional point in CommCare proficiency score increased the quality score by around half a point (0.541, P=.07), and experience score by around a third of a point (0.308, P=.03). Age affected CommCare user type negatively, with an increase in age increasing the likelihood of belonging to a lower category of CommCare adoption (-0.105, P=.08). Other individual characteristics did not affect adoption or the predicted values estimating the relationship between adoption and quality and experience of care, although illiteracy was able to affect the relationship negatively. CONCLUSIONS: mHealth technology adoption by frontline workers can positively impact the quality and experience of care they provide. Individual characteristics, especially literacy and age, can be important elements affecting technology adoption and the way users leverage the technology for their work. Our formative study provides informed hypotheses and methods for further research.

Using electronic technology to improve clinical care - results from a before-after cluster trial to evaluate assessment and classification of sick children according to Integrated Management of Childhood Illness (IMCI) protocol in Tanzania.

BACKGROUND: Poor adherence to the Integrated Management of Childhood Illness (IMCI) protocol reduces the potential impact on under-five morbidity and mortality. Electronic technology could improve adherence; however there are few studies demonstrating the benefits of such technology in a resource-poor settings. This study estimates the impact of electronic technology on adherence to the IMCI protocols as compared to the current paper-based protocols in Tanzania. METHODS: In four districts in Tanzania, 18 clinics were randomly selected for inclusion. At each site, observers documented critical parts of the clinical assessment of children aged 2 months to 5 years. The first set of observations occurred during examination of children using paper-based IMCI (pIMCI) and the next set of observations occurred during examination using the electronic IMCI (eIMCI). Children were re-examined by an IMCI expert and the diagnoses were compared. A total of 1221 children (671 paper, 550 electronic) were observed. RESULTS: For all ten critical IMCI items included in both systems, adherence to the protocol was greater for eIMCI than for pIMCI. The proportion assessed under pIMCI ranged from 61% to 98% compared to 92% to 100% under eIMCI (p < 0.05 for each of the ten assessment items). CONCLUSIONS: Use of electronic systems improved the completeness of assessment of children with acute illness in Tanzania. With the before-after nature of the design, potential for temporal confounding is the primary limitation. However, the data collection for both phases occurred over a short period (one month) and so temporal confounding was expected to be minimal. The results suggest that the use of electronic IMCI protocols can improve the completeness and consistency of clinical assessments and future studies will examine the long-term health and health systems impact of eIMCI.

A pilot study of an mHealth application for healthcare workers: poor uptake despite high reported acceptability at a rural South African community-based MDR-TB treatment program.

INTRODUCTION: As the South African province of KwaZulu-Natal addresses a growing multidrug-resistant tuberculosis (MDR-TB) epidemic by shifting care and treatment from trained specialty centers to community hospitals, delivering and monitoring MDR-TB therapy has presented new challenges. In particular, tracking and reporting adverse clinical events have been difficult for mobile healthcare workers (HCWs), trained health professionals who travel daily to patient homes to administer and monitor therapy. We designed and piloted a mobile phone application (Mobilize) for mobile HCWs that electronically standardized the recording and tracking of MDR-TB patients on low-cost, functional phones. OBJECTIVE: We assess the acceptability and feasibility of using Mobilize to record and submit adverse events forms weekly during the intensive phase of MDR-TB therapy and evaluate mobile HCW perceptions throughout the pilot period. METHODS: All five mobile HCWs at one site were trained and provided with phones. Utilizing a mixed-methods evaluation, mobile HCWs' usage patterns were tracked electronically for seven months and analyzed. Qualitative focus groups and questionnaires were designed to understand the impact of mobile phone technology on the work environment. RESULTS: Mobile HCWs submitted nine of 33 (27%) expected adverse events forms, conflicting with qualitative results in which mobile HCWs stated that Mobilize improved adverse events communication, helped their daily workflow, and could be successfully expanded to other health interventions. When presented with the conflict between their expressed views and actual practice, mobile HCWs cited forgetfulness and believed patients should take more responsibility for their own care. DISCUSSION: This pilot experience demonstrated poor uptake by HCWs despite positive responses to using mHealth. Though our results should be interpreted cautiously because of the small number of mobile HCWs and MDR-TB patients in this study, we recommend carefully exploring the motivations of HCWs and technologic enhancements prior to scaling new mHealth initiatives in resource poor settings.

Electronic decision protocols for ART patient triaging to expand access to HIV treatment in South Africa: a cross sectional study for development and validation.

BACKGROUND: The shortage of doctors and nurses, along with future expansion into rural clinics, will require that the majority of clinic visits by HIV infected patients on antiretroviral therapy (ART) are managed by non-doctors. The goal of this study was to develop and evaluate a screening protocol to determine which patients needed a full clinical assessment and which patients were stable enough to receive their medications without a doctor's consultation. For this study, we developed an electronic, handheld tool to guide non-physician counselors through screening questions. METHODS: Patients visiting two ART clinics in South Africa for routine follow-up visits between March 2007 and April 2008 were included in our study. Each patient was screened by non-physician counselors using the handheld device and then received a full clinical assessment. Clinicians' report on whether full clinical assessment had been necessary was used as the gold standard for determining "required referral". Observations were randomly divided into two datasets--989 for developing a referral protocol and 200 for validating protocol performance. RESULTS: A third of patients had at least one physical complaint, and 16% had five or more physical complaints. 38% of patients required referral for full clinical assessment. We identify a subset of questions which are 87% sensitive and 47% specific for recommended patient referral. CONCLUSIONS: The final screening protocol is highly sensitive and could reduce burden on ART clinicians by 30%. The uptake and acceptance of the handheld tool to support implementation of the protocol was high. Further examination of the data reveals several important questions to include in future referral algorithms to improve sensitivity and specificity. Based on these results, we identify a refined algorithm to explore in future evaluations.

Mobile phone tools for field-based health care workers in low-income countries.

In low-income regions, mobile phone-based tools can improve the scope and efficiency of field health workers. They can also address challenges in monitoring and supervising a large number of geographically distributed health workers. Several tools have been built and deployed in the field, but little comparison has been done to help understand their effectiveness. This is largely because no framework exists in which to analyze the different ways in which the tools help strengthen existing health systems. In this article we highlight 6 key functions that health systems currently perform where mobile tools can provide the most benefit. Using these 6 health system functions, we compare existing applications for community health workers, an important class of field health workers who use these technologies, and discuss common challenges and lessons learned about deploying mobile tools.

Comprehensive yet scalable health information systems for low resource settings: a collaborative effort in sierra leone.

We address the problem of how to integrate health information systems in low-income African countries in which technical infrastructure and human resources vary wildly within countries. We describe a set of tools to meet the needs of different service areas including managing aggregate indicators, patient level record systems, and mobile tools for community outreach. We present the case of Sierra Leone and use this case to motivate and illustrate an architecture that allows us to provide services at each level of the health system (national, regional, facility and community) and provide different configurations of the tools as appropriate for the individual area. Finally, we present a, collaborative implementation of this approach in Sierra Leone.

The OpenMRS Implementers Network.

OBJECTIVE: OpenMRS (www.openmrs.org) is a configurable open source electronic medical record application developed and maintained by a large network of open source developers coordinated by the Regenstrief Institute and Partners in Health and mainly used for HIV patient and treatment information management in Africa. Our objective is to develop an open Implementers Network for OpenMRS to provide regional support for the growing number of OpenMRS implementations in Africa and to include African developers and implementers in the future growth of OpenMRS. METHODS: We have developed the OpenMRS Implementers Network using a dedicated Wiki site and e-mail server. We have also organized annual meetings in South Africa and regional training courses at African locations where OpenMRS is being implemented. An OpenMRS Internship program has been initiated and we have started collaborating with similar networks and projects working in Africa. To evaluate its potential, OpenMRS was implemented initially at one site in South Africa by a single implementer using a downloadable OpenMRS application and only the OpenMRS Implementers Network for support. RESULTS: The OpenMRS Implementers Network Wiki and list server have grown into effective means of providing implementation support and forums for exchange of implementation experiences. The annual OpenMRS Implementers meeting has been held in South Africa for the past three years and is attracting successively larger numbers of participants with almost 200 implementers and developers attending the 2008 meeting in Durban, South Africa. Six African developers are presently registered on the first intake of the OpenMRS Internship program. Successful collaborations have been started with several African developer groups and projects initiated to develop interoperability between OpenMRS and various applications. The South African OpenMRS Implementer group successfully configured, installed and maintained an integrated HIV/TB OpenMRS application without significant programming support. Since then, this model has been replicated in several other African sites. The OpenMRS Implementers Network has contributed substantially to the growth and sustainability of OpenMRS in Africa and has become a useful way of including Africans in the development and implementation of OpenMRS in developing countries. The Network provides valuable support and enables a basic OpenMRS application to be implemented in the absence of onsite programmers.

Experience in implementing the OpenMRS medical record system to support HIV treatment in Rwanda.

The challenge of scaling up HIV treatment in Africa has led to a new emphasis on improving health systems in impoverished areas. One aspect of this is the development and deployment of electronic medical record systems to support HIV and TB treatment. In this paper we describe the design and implementation of a new medical record architecture to support an HIV treatment program in rural Rwanda. The architecture is called OpenMRS and it has been developed to address the problem of configuring EMR systems to suit new sites, languages and diseases. OpenMRS uses a data dictionary called the concept dictionary to represent all the possible data items that can be collected. This allows new items to be added to the system by non-programmers. In addition, there are form creation tools that use drag and drop web technologies to simplify form construction. The OpenMRS system was first implemented in Kenya in February 2006 and then in Rwanda in August 2006. The system is now functioning well and we are developing extensions to improve the support for the clinic. These include improved, easy to use reporting tools, support for additional clinical problems including nutrition and child health, better database synchronization tools, and modules to collect laboratory data and support the pharmacy. The system is also in use in South Africa and Lesotho and is being deployed in Tanzania and Uganda.

Rapid deployment of electronic medical records for ARV rollout in rural Rwanda.

While most people with AIDS do not yet have access to anti-retroviral drugs (ARVs), large ARV treatment programs are being rolled out in many areas in Sub-Saharan Africa. ARV programs have substantial data management needs, which electronic medical record systems (EMRs) are helping to address. While most sophisticated EMRs in low-income regions are in large cities, where infrastructure and staffing needs are more easily met, Partners In Health (PIH) has pioneered web-based EMRs for HIV and TB treatment in rural areas. The HIV-EMR, developed in Haiti [1], was de-ployed in two Rwandan health districts starting in Au-gust 2005. The addition of new features and adaptation to local needs is happening concurrently with the rapid scale-up and evolution of the medical program itself.